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Software-Intensive  Programs: 
Productivity  and  Policy 


•  Software  acquisition:  a  well-publicized  problem  across  the 
Department  of  Defense  .DoD, 

-  General  Accounting  Office  (GAO)  (2009):  large-scale  software  acquisitions 
falling  short  of  cost,  schedule,  and  performance  goals 

•  With  sophisticated  tools  and  ca  abilities  shouldn't  the  work  be 
getting  easier? 

*  Development:  much  more  than  writing  code 

*  Complicating  factors: 

-  Lack  of  formal  requirements  definition 

-  Misunderstandings  between  user  and  developer 

-  Requirements  creep 

-  Technology  insertions  and  upgrades 

-  Inspections  and  testing 

-  Volume  of  scrap  and  rework 

-  Policy  changes 
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Software-Intensive  Programs:  Current  Policy 


*  MIL-STD-498:  Uniform  requirements  for  development  and 
documentation  (22  data  item  descriptions) 

*  DoDI  5000.02  and  DoD  5000.04-M-l:  Software  Resources  Data 
Report  reporting  (via  Defense  Cost  and  Resource  Center) 

*  Software  process  improvement  initiatives,  driven  by  Section 
804  of  National  Defense  Authorization  Act  (2003),  and 
promulgated  by  ASN(RDA) 

*  Some  apply  solely  to  Acquisition  Category  (ACAT)  I  programs; 
most  are  implemented  at  contract  level 


Complex  web  of  policy,  regulations,  instructions,  and  best  practices 
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Software-Intensive  Programs:  Research  Questions 


*  How  can  the  DoD  adjust  its  acquisition  processes  to  improve 
developers'  productivity? 


*  How  can  the  DoD  effectively  reward  desired  behaviors? 


4 


DAHLGREN 


Software-Intensive  Programs:  Study  Overview 


*  Criteria  for  sample:  ACAT I  and  software  intensive 

*  Stratify  programs  by  level  of  success 

*  Identify  best  practices 

-  Literature  review 

-  Subject-matter  expert  inquiries 

*  Consider  "success"  metrics  (cost,  schedule,  performance)  in 
light  of 

-  Development  approaches 

-  Management  styles 

-  Organizational  policy  and  best  practices 

-  DoD  policy  and  best  practices 

*  Findings  and  recommenaations 
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Methodology:  Successful  Programs 


*  Extracted  data  on  Navy  and  Marine  Corps  Major  Defense 
Acquisition  Programs  from  Defense  Acquisition  Management 
Information  Retrieval  service 

-  Filter  #1:  ACAT  I  or  special  interest 

-  Filter#2:  No  Nunn-McCurdy  breaches 

*  Unless  Average  Procurement  Unit  Cost  breach  driven  by  changes  to 
quantity 

-  Filter  #3:  Currency— active  program  (April  2011) 

-  Filter  #4:  History— at  least  three  years  into  development 

-  Filter  #5:  Software  intensive  (>  $20M  in  software) 


Data  on  "successful"  programs  gathered  from  interviews  and  public  domain  sources 
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Successful  Software-Intensive  Programs 


EA-18G  ("Growler")  Electronic  Warfare  Aircraft 

Navy  Multiband  Terminal 

AGM-88E  Anti-Radiation  Guided  Missile 

P-8A  Multimission  Maritime  Aircraft 

Standard  Missile  (SM)-6 

Cooperative  Engagement  Capability  (CEC) 
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P-8A  Multimission  Maritime  Aircraft 
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U.S.  Navy-released  photo 


*  Replaced  P-3C  Orion  capabilities 

*  Open  architecture  approach 

*  Detailed  planning  phase;  robust 
Analysis  of  Alternatives  (AoAs) 

*  wen-Daiancea  package  of  cost, 
schedule,  and  performance 
(Independent  Cost  Estimate 
added  funds  for  software 
development) 

*  Good  relationship  with 
requirements  community; 
program  could  speak  in  unison 
and  maintain  executability 

*  Heightened  sense  of  immediacy 


8 


DAHLGREN 


Standard  Missile  6  (SM-6) 


Next-generation  SM  with  extended 
range  and  active  missile-seeker 
homing  capabilities;  capable  of 
responding  to  various  threats 

urgent  need  for  successful  program 

Thorough  pre-Milestone  B  planning 

Strong  ASN.RDA,  support:  full 
funding  based  on  a  realistic,  risk- 
based  cost  estimate 

Prior  experience  helped  manage 
anticipated  pitfalls 

Positive  relationship  with  industry 
neipea  ensure  quality  staffing 
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Cooperative  Engagement  Capability  (CEC) 


AEGIS  TAMD 


U.S.  Navy-released  graphic 


Operational  or  Study 
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Engagement  on 
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■  Cued  AMRAAM  self  defense 


Integrated  battle-force  combat 
systems  and  sensors;  data 
distribution  for  a  common  composite 
track  database;  enhanced  ship 
self-defense  capability 

Classic  acquisition  processes  through 
mid-1990s,  when  acquisition  reform 
impacted  strategy 

Strong  mission  focus  and  leadership 

Well-balanced,  experienced  team 

Adapted  to  changing  labor  market 
(Ada  —  >  C++) 
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Successful  Programs:  Findings 


*  Experience  to  identify  and  mitigate  risks 

*  Leadership  continuity  (including  senior  engineers) 

*  Communication  skills  (candor  and  honesty) 

*  Empowerment  of  program  managers  to  make  good  decisions 

*  Sound  knowledge-based  business  plan  at  outset 

*  Clear,  well-defined  requirements 

*  used  mature  technologies  and/or  production  techniques 

*  Established  realistic,  risk-based  cost  and  schedule 

*  Disciplined  execution  with  resistance  to  new  requirements 

*  Stabilization  of  funding  and  resources 

*  Contractual  incentives  and  stalwart  government  review  team 

*  Holistic  approach  to  contracting  (system  ->  platform  ->) 
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Environmental  Factors  (Future  Research) 


*  Technology  levels 

*  Stability  of  requirements 

*  Available  time  and  approach  (incremental  vs.  one  shot) 

*  Personnel  and/or  developer  attributes 

*  Capabilities  of  analysts  and  programmers 

*  Application  domain  experience 

*  Continuity  of  personnel  and/or  learning  curve 

*  Modern  practices  and  tools 

*  Team  organization  and  communication 
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Concluding  Thoughts:  Weapon  Systems 
Acquisition  Reform  Act  (WSARA)  and  Beyond 


*  WSARA  aspires  for  more  executable  programs  ... 

-  Supplant  risk  with  knowledge 

-  Milestone  A  cost  estimate  with  confidence  levels 

-  Competitive  prototypes 

-  Enhanced  requirements  for  AoAs 

-  Pre-Milestone  B  review  of  technology  maturity  and  integration  risk 

-  Pre-Milestone  B  Preliminary  Design  Review 

-  Renewed  emphasis  on  systems  engineering  and  testing 

-  Configuration  steering  boards  to  stabilize  requirements 

-  Post-Critical  Design  Review  assessment  of  progress 

*  GAO  (2010:  modest  improvements;  increased  knowledge  at 
key  decision  points 

*  GAO  (2011):  "meaningful  steps"  taken  to  reprioritize  and 
rebalance  portfolio,  but  still  a  factor  on  GAO  High-Risk  List 
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